Database syncing

ABSTRACT

A method for database syncing is provided. The method includes receiving database change indicators indicating changes to a database. For each database change indicator, the method also includes determining whether the respective change to the database corresponds to a single change to a single version of the database, one of multiple changes to a common version of the database, or one of multiple changes to different versions of the database. The method further includes generating a change log compaction data structure comprising entries. Each entry includes an exact entry defining a corresponding single change to a corresponding single version of the database, a range entry defining corresponding multiple changes to a corresponding common version of the database, or an interval entry defining corresponding multiple changes to corresponding different versions of the database.

TECHNICAL FIELD

This disclosure relates to syncing databases.

BACKGROUND

Databases are a ubiquitous part of data storage. Databases are generallyresponsible for many services used every day within information systems.Databases have an organized structure that stores data and allows thestored data to be accessed or queried. The organized structure may formrelationships between data within a database for database users toreference related data. As some databases change or update over time,database users often seek to synchronize to these modified databases andto understand such modifications.

SUMMARY

One aspect of the disclosure provides a method for database syncing. Themethod includes receiving, at data processing hardware, database changeindicators indicating changes to a database. For each database changeindicator, the method also includes determining, by the data processinghardware, whether the respective change to the database corresponds to:a single change to a single version of the database; one of multiplechanges to a common version of the database; or one of multiple changesto different versions of the database. The method further includesgenerating, by the data processing hardware, a change log compactiondata structure including entries. Each entry includes an exact entrydefining a corresponding single change to a corresponding single versionof the database, a range entry defining corresponding multiple changesto a corresponding common version of the database, or an interval entrydefining corresponding multiple changes to corresponding differentversions of the database.

Implementations of the disclosure may include one or more of thefollowing optional features. In some implementations, the change logcompaction data structure includes an index space and a data space, anynegative space of the index space includes known space having no changesto the database. Optionally, the database change indicator may include acommit log entry. Additionally or alternatively, the different versionsof the database corresponding to the interval entry may span acontinuous interval of time.

Another aspect of the disclosure provides a method for database syncing.The method includes receiving, at data processing hardware, a query forchanges to a database from a client, the query indicating a set of dataranges and an interval of time. The method also includes accessing, bythe data processing hardware, a change log compaction data structureincluding entries. Each entry includes: an exact entry defining acorresponding single change to a corresponding single version of thedatabase; a range entry defining corresponding multiple changes to acorresponding common version of the database; or an interval entrydefining corresponding multiple changes to corresponding differentversions of the database. The method further includes determining, bythe data processing hardware, whether any entry of the change logcompaction data structure satisfies the query. When at least one entryof the change log compaction data structure satisfies the query, themethod includes returning, by the data processing hardware, thecorresponding at least one entry to the client.

This aspect may include one or more of the following optional features.In some examples, when at least one entry of the change log compactiondata structure satisfies the query, the method includes returning, bythe data processing hardware, at least one of the following: a databasechange indicator, a database change log entry, or a change, thatcorresponds to the at least one entry of the client. In someimplementations, when at least one entry of the change log compactiondata structure satisfies the query, the method includes returning, bythe data processing hardware, a bloom filter corresponding to the atleast one entry of the client.

In some configurations, the change log compaction data structureincludes an index space and a data space. Here, any negative space ofthe index space includes known space having no changes to the database.When at least one entry of the change log compaction data structuresatisfies the query, the method may include returning, by the dataprocessing hardware, any overlapping entries of the index space of theat least one entry satisfying the query, at or following the at leastone entry satisfying the query. Additionally or alternatively, whenmultiple entries of the change log compaction data structure satisfy thequery, the method may include returning, by the data processinghardware, the corresponding entries to the client as a range sorted by astart of the range in the index space or the data space.

In some implementations, the method includes determining, by the dataprocessing hardware, whether any entry of the change log compaction datastructure satisfying the query includes an interval entry. For eachinterval entry satisfying the query, the method may include returning aspan defined as a set of all corresponding versions of the databasedefined by the interval entry, the span having a start version of thedatabase and an end version of the database.

Another aspect of the disclosure provides a system for database syncing.The system includes data processing hardware and memory hardware incommunication with the data processing hardware. The memory hardwarestores instructions that when executed on the data processing hardwarecause the data processing hardware to perform operations. The operationsinclude receiving database change indicators indicating changes to adatabase. For each database change indicator, the operations includedetermining, whether the respective change to the database correspondsto a single change to a single version of the database, one of multiplechanges to a common version of the database, or one of multiple changesto different versions of the database. The operations also includegenerating a change log compaction data structure including entries.Each entry includes an exact entry defining a corresponding singlechange to a corresponding single version of the database, a range entrydefining corresponding multiple changes to a corresponding commonversion of the database, or an interval entry defining correspondingmultiple changes to corresponding different versions of the database.

Implementation of the disclosure may include one or more of thefollowing optional features. In some configurations, the change logcompaction data structure includes an index space and a data space.Here, any negative space of the index space includes known space havingno changes to the database. In some examples, the database changeindicator includes a commit log entry. Additionally or alternatively,the different versions of the database corresponding to the intervalentry may span a continuous interval of time.

Yet another aspect of the disclosure provides a system for syncingdatabases. The system includes data processing hardware and memoryhardware in communication with the data processing hardware. The memoryhardware stores instructions that when executed on the data processinghardware cause the data processing hardware to perform operations. Theoperations include receiving a query for changes to a database from aclient. The query indicates a set of data ranges and an interval oftime. The operations also include accessing a change log compaction datastructure which includes entries. Each entry includes an exact entrydefining a corresponding single change to a corresponding single versionof the database, a range entry defining corresponding multiple changesto a corresponding common version of the database, or an interval entrydefining corresponding multiple changes to corresponding differentversions of the database. The operations also include determiningwhether any entry of the change log compaction data structure satisfiesthe query. When at least one entry of the change log compaction datastructure satisfies the query, the operations include returning thecorresponding at least one entry to the client.

Implementations of this aspect may include one or more of the followingoptional features. In some implementations, when at least one entry ofthe change log compaction data structure satisfies the query, theoperations include returning at least one of a database changeindicator, a database change log entry, or a change corresponding to theat least one entry of the client. In some implementations, when at leastone entry of the change log compaction data structure satisfies thequery, the system includes returning a bloom filter corresponding to theat least one entry of the client.

In some examples, the change log compaction data structure includes anindex space and a data space. In this example, any negative space of theindex space includes known space having no changes to the database. Whenat least one entry of the change log compaction data structure satisfiesthe query, the operation may include returning any overlapping entriesof the index space of the at least one entry satisfying the query, at orfollowing the at least one entry satisfying the query. Additionally oralternatively, when multiple entries of the change log compaction datastructure satisfy the query, the operations may include returning thecorresponding entries to the client as a range sorted by a start of therange in the index space or the data space.

In some configurations, the operations include determining whether anyentry of the change log compaction data structure satisfying the queryincludes an interval entry. Here, for each interval entry satisfying thequery, the operations include returning a span defined as a set of allcorresponding versions of the database defined by the interval entry,the span having a start version of the database and an end version ofthe database.

The details of one or more implementations of the disclosure are setforth in the accompanying drawings and the description below. Otheraspects, features, and advantages will be apparent from the descriptionand drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIGS. 1A and 1B are schematic views of example database synchronizationenvironments.

FIG. 2 is a schematic view of an example database logger within adatabase synchronization environment.

FIGS. 3A-3C are schematic views of example compacters within a databasesynchronization environment.

FIGS. 4A-4D are schematic views of example queries within a databasesynchronization environment.

FIGS. 5 and 6 are block diagrams of example arrangements of operationsperformed within the database synchronization environment.

FIG. 7 is a schematic view of an example computing device that may beused to implement the systems and methods described herein.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

Today computing devices often rely on databases. These databases mayinclude large amounts of data that may change and mutate over time. Asthe data within a database changes, the device and/or client of thedatabase will need to synchronize with the changes. Rather than acquirea significant portion of the database each time a device attempts tosynchronize with a database, devices may be generally configured tosolely identify changes that have occurred since the last update.Although this approach reduces the amount of data within a database adevice may have to receive, the approach may remain inefficient. Forexample, the device may have to parse through the entire databaselooking for changes since the last update. Some approaches for examplecreate markers within the database corresponding to each change thatoccurs. Yet over time and large amounts of data, even markers becomeincreasingly inefficient. Therefore, it is desirable to provide acompact space conserving data structure to more effectively and moreefficiently synchronize with a database.

Referring to FIG. 1, in some implementations, an example databasesynchronization environment 10 includes one or more client devices 102,102 a-n, each associated with a respective client 20 in communicationwith a database synchronization system 100 of a remote system 110 via anetwork 120. The remote system 110 may be a distributed system (e.g., acloud environment) having scalable/elastic computing resources 112(e.g., data processing hardware) and/or storage resources 114 (e.g.,memory hardware). The database synchronization system 100 is configuredto synchronize information and/or data regarding a database 130, such asa distributed database 130 of the remote system 110 (e.g., FIG. 1).Although, the disclosure illustrates the database synchronization system100 with respect to a distributed system (e.g., the cloud environment),the database synchronization system 100 may interact with any type ofdatabase. Some examples of other types of databases 130 includerelational databases (e.g., structured query language database),non-structured query language databases (e.g., a document database, agraph database, a key-value database, or a table-style database), orobject-oriented databases. Additionally or alternatively, the database130 may be more than one database of the same database type or more thanone database of different types of databases.

As illustrated by a dotted line enclosure in FIG. 1A, the databasesynchronization system 100 generally includes a database logger 200 anda compacter 300. To synchronize information and/or data regarding thedatabase 130, the database synchronization system 100 tracks changes tothe database 130 and constructs a data structure based on these trackedchanges. The database logger 200 is configured to track changes to thedatabase 130 and to communicate the tracked changes to the compacter300. With the communicated changes from the database logger 200, thecompacter 300 constructs the data structure based on the changes. Insome examples, the database 130 is additionally integrated into thesynchronization system 100. For example, FIG. 1A illustrates thedatabase 130 as optional to the database synchronization system 100 by adotted box around the database 130. Integrating the database 130 withthe database synchronization system 100 may, depending on theconfiguration of the database synchronization environment 10, reducecommunication time between the database synchronization system 100 andthe database 130. In some implementations, the database synchronizationsystem 100 is a database management system or part of a databasemanagement system. This management system may be local to the database130 or remotely communicate with the database 130.

Referring to FIG. 1B, a client 20, via a client device 102, sends aquery 140 to the database synchronization system 100. The query 140refers to a request by the client 20 to extract data from the database130. Here, by the query 140, the client 20 seeks to synchronize theclient device 102 with data contained within the database 130.Synchronization generally refers to a process of updating or catching upthe client 20 to changes that have occurred within the database 130since the client 20, via the client device 102, last communicated withthe database 130. As an example, a client 20 using an electronic mail(email) application synchronizes with an email database 130corresponding to a cloud based email server 110. In this example, theclient 20 synchronizes with the email database 130 by opening the emailapplication on a client device 102, such as a mobile phone, or byselecting to send and to receive email within the email application. Inother words, the client 20 synchronizes with the email database 130 by aquery 140 to send and to receive email. For example, FIG. 1B depicts thequery 140 as asking the database synchronization system 100 if therehave been any “changes since 9:47 A.M. and the current time.” In thisexample, the client 20 last updated (i.e. synchronized) his or her emailat 9:47 A.M. and wants to know if there have been any updates to his orher email between the current time of the query 140 and the last updateat 9:47 A.M. As part of the synchronization, the databasesynchronization system 100 identifies whether any updates (e.g., newemails) have occurred by determining whether there have been any changes(e.g., additions of data such as email) to the email database 130. Basedon this determination, the database synchronization system 100communicates a response 150 that there has been “no new email since 9:47A.M.”

FIG. 2 provides further detail to the database logger 200 of thedatabase synchronization system 100. FIG. 2 includes an example of thedatabase 130 over a progression of time. As illustrated by FIG. 2,databases, such as database 130, include a dataspace 132. A dataspacegenerally refers to a space accessible for or occupied with data D. Forexample, FIG. 2 depicts database 130 with a simplified dataspace havingeight spaces for data D. Over a progression of time, different versions“t” of the database 130 may capture changes (e.g., write functions ordelete functions) made to the dataspace 132. A version of the database130 is the database 130 at a particular moment of time such that eachversion t represents a different moment in time for the database 130. Inother words, versions t_(n) of the database 130 correspond to snapshotsof the database 130 at discrete intervals of time. Each version t mayhave an associated time stamp to help identify the version t. In someexamples, the version t of the database 130 changes according to a clockof a processor associated with the database 130 such that eachincremental version t_((n+1)) is an increment of the timeclock. In otherexamples, the version t of the database 130 increments when there is achange to the database 130. In yet other examples, each version tincrements according to a preprogrammed interval of time.

FIG. 2 depicts six versions t₀-t₅ of the database 130. Each version tincludes a representation of the dataspace 132 of the database 130.Between each version t of the database 130, changes may occur within thedataspace 132. Each change, such as an addition or a deletion, occurswhen a transaction is committed. When a transaction is committed (i.e. achange occurs), the transaction is configured to be memorialized as achange log entry 210 in a database change log 220 (i.e. a commit log ora transaction log). The change log entry 210 may be a line of textrelated to the transaction, an entry of the actual transaction, or anyother representation that signifies or provides a reflection of thechange to the database 130 corresponding to the transaction.Additionally or alternatively, each change log entry 210 includes theversion t of the database 130 associated with the change.

In some implementations, the database logger 200 includes the databasechange log 220 with the corresponding change log entries 210. In someexamples, the database logger 200 includes more than one database changelog 220 corresponding to more than one database 130. In other examples,the database change logger 200 includes a single database change log 220for more than one database 130. For simplicity FIG. 2 depicts a singledatabase change log 220 for the database 130.

Referring further to FIG. 2, at a first version t₀, data D₁-D₅ occupiesthe database 130. As the first version t₀ of the database 130 depictedin the example of FIG. 2, no change has yet occurred to the database130. At a second version t₁, data D₁ has been deleted from the database130 and data D₆ has been written to the database 130. A first change logentry 210 a memorializes the deletion of data D₁ and the addition ofdata D₆ in the database change log 220. At a third version t₂ of thedatabase 130, data D₂-D₆ within the dataspace 132 does not change.Without changes, the database logger 200 does not generate a databasechange log entry 210. At a fourth version t₃, data D₁ has been writtento the database 130 and data D₆ has been deleted from the database 130.A second change log entry 210 b memorializes the addition of data D₁ andthe deletion of data D₆ in the database change log 220. At a fifthversion t₄ of the database 130, data D₂ has been deleted and a thirdchange log entry 210 c memorializes the deletion of data D₂. At a sixthversion t₅, data D₁ has been deleted and data D₆ and D₇ has beenwritten. The database change log 220 includes a fourth entry 210 d torepresent the changes during the sixth version t₅. With each change of achange log entry 210, the database logger 200 may generate a databasechange indicator 230 corresponding to the change to communicate to thecompacter 300. For example, the first change log entry 210 a and thesecond change log entry 210 b each correspond to two database changeindicators 230, 230 a ₁₋₂ and 230, 230 b ₁₋₂ respectively, while thefourth change log entry 210 d corresponds to three database change logindicators 230, 230 d ₁₋₃.

FIGS. 3A-3C are examples of the compacter 300 of the databasesynchronization system 100. The compacter 300 is configured to receivedatabase change indicators 230 indicating changes to the database 130from the database logger 200. Depending on the change log entry 210, thedatabase change indicator 230 may be identical to the change log entry210 (e.g., a commit log entry) or different from the change log entry210. Configurations with change log entries 210 predominantly similar tothe database change indicators 230 may reduce additional processing forthe database logger 200 and potentially reduce computationalredundancies or risks of error. For each received database changeindicator 230, the compacter 300 determines the type of underlyingchange to the database 130. In some implementations, the compacter 300includes a change determiner 310 to determine whether the respectivechange to the database 130 corresponds to a single change 312 to asingle version t_(i) of the database 130, one of multiple changes 314 cto a common version t_(i) of the database 130, or one of multiplechanges 314 d to different versions t_(i-n) of the database 130. Forexample, as depicted in FIGS. 3A and 3B, the database change indicator230 c corresponds to a single change 312 to the fifth version t₄ of thedatabase 230. In FIGS. 3A and 3B, the change determiner 310 alsodetermines that several change indicators 230 correspond to one ofmultiple changes 314 c to a common version t_(i) of the database 130.Database change indicators 230 a ₁ and 230 a 2 correspond to multiplechanges 314 c to the second version t₁ of the database 130. Databasechange indicators 230 b ₁ and 230 b 2 correspond to multiple changes 314c to the fourth version t₃ of the database 130. Database changeindicators 230 d ₁₋₃ correspond to multiple changes 314 c to the sixthversion t₅ of the database 130. Here, a common version refers to changesthat occur (i.e. share) during the same version. In other words, thesechanges have the same version in common. Moreover, collectively,database change indicators 230 b-d correspond to multiple changes 314 dto different versions, such as the fourth version t₃, the fifth versiont₄, and the sixth version t₅.

Referring to FIGS. 3A-3C, the compacter 300 further includes aconstructer 320 configured to generate entries 322 that form a changelog compaction data structure 324. In some examples, the constructer 320may generate an exact entry 322 e, a range entry 322 r, or an intervalentry 322 i. Each entry 322 corresponds to the different types ofchanges 312, 314 determined by the change determiner 310 based on thedatabase change indicators 230. The exact entry 322 e defines acorresponding single change 312 to a corresponding single version t_(i)of the database 130. The range entry 322 r defines correspondingmultiple changes 314 c to a corresponding common version t_(i) of thedatabase 130. The interval entry 322 i defines corresponding multiplechanges 314 d to corresponding different versions of the database 130.

In some implementations, the change log compaction data structure 324includes a version space 326 v and a data space 326 d. The version space326 v enables the constructer 320 to build the change log compactiondata structure 324 such that a query 140 of the change log compactiondata structure 324 by the client 20 may efficiently focus on aparticular span of versions t_(i) (e.g., versions since previoussynchronization). The data space 326 d, much like the data space 132 ofthe database 130, corresponds to space accessible for or occupied withentries 322 (i.e. compaction data) within the change log compaction datastructure 324. In the examples depicted throughout FIGS. 3A-3C, thechange log compaction data structure 324 generally includes the versionspace 326 v and the data space 326 d, but may also include an indexspace 326 i (e.g., FIGS. 3B and 3C). Generally speaking, an index ispart of a data structure and may optimize searching (e.g., rapid lookup)and accessing of data within the data structure. An index space includesindexes that are unique identifiers, such as a number or a time stamp.The index space 326 i may also include a pointer to the underlying dataD within the data space 326 d that corresponds to the index (i.e. thedata that is uniquely identified by the index). Additionally oralternatively, the change log compaction data structure 324 isconfigured for secondary indexes, which may or may not modify the datachanges. Because the change log compaction data structure 324corresponds to changes over the data space 326 d, version space 326 v,and/or index space 326 i, synchronization with a client 20 roughlyscales with the number of changes to a corpus, rather than a size of thecorpus (e.g., size of a database). Furthermore, the synchronizationsystem 100 is flexible such that a change log compaction data structure324 may be constructed at any time (e.g., rather than tied to a time oforigination for a database) and the change log compaction data structureis not necessarily a critical dependency in a write path for thedatabase 130. An advantage of the flexibility of the synchronizationsystem 100 is that because the change log compaction data structure 324may be constructed at any time, the change log compaction data structure324 does not require durable storage.

In some examples, such as FIG. 3B, the constructer 320 generates asingle change log compaction data structure 324 that includes the dataspace 326 d, the index space 326 i, and the version space 326 v. Inother examples, such as FIG. 3C, the constructer 320 is configured togenerate more than one change log compaction data structure 324. Forexample, the constructer 320 generates a first change log compactiondata structure 324 a with a dataspace 326 d and a version space 326 vand a second change log compaction data structure 324 b with an indexspace 326 i and a version space 326 v.

Referring further to FIGS. 3A-3C, the constructer 320 interprets thechanges 312, 314 c, 314 d from the change determiner 310 and generatesthe change log compaction data structure 324. In these examples, thesingle change 312 corresponding to the database change indicator 230 cgenerates an exact entry 322 e at the fifth version t₄ within the changelog compaction data structure 324. The database change indicators 230 a₁₋₂, 230 b ₁₋₂, 230 d ₁₋₃ corresponding to multiple changes 314 c to acommon version (e.g., the second version t₁, the fourth version t₃, andthe sixth version t₅, respectfully) each generate a range entry 322 r:the database change indicators 230 a ₁₋₂ of the second version t₁generate range entry 322 r ₁; the database change indicators 230 b ₁₋₂of the fourth version t₃ generate range entry 322 r ₂; and the databasechange indicators 230 d ₁₋₃ of the sixth version t₅ generate range entry322 r ₃. The database change indicators 230 b-d also correspond tomultiple changes 314 d to different versions of the database 130 (e.g.,the fourth version t₃, the fifth version t₄, and the sixth version t₅,respectfully) and generate an interval entry 322 i. Here, like someconfigurations, the interval entry 322 i spans a continuous interval oftime relating to different versions, such as versions t₃-t₅. In someexamples, entries 322 include metadata, such as an update time or adelete time, corresponding to the underlying change or database changeindicator 230. In some implementations, a bloom filter may filter themetadata of the entries 322 which, for example, may later be optimizedfor secondary indexes. In some configurations, when entries 322correspond to a delete change, a bloom filter or a existence filteraccumulates these changes. This bloom filter or existence filter maythen be sent to a client 20 as a response 150 to a query 140.Additionally or alternatively, a bloom filter may be utilized to reviewdata related to an entry 322 and/or to identify the underlying changecorresponding to an entry 322.

In some implementations, the constructer 320 constructs the change logcompaction data structure 324 such that any negative space 328 isactually known space having no changes to the database 130. When thenegative space 328 is actually known space having no changes to thedatabase 130, synchronization may confidently skip large ranges of datawithout risking missing data. In other words, navigating through thedata log compaction data structure 324 may inherently skip large rangesof data because the data log compaction data structure 324 may includeonly entries 322 (e.g., range entries 322 r, interval entries 322 i, orexact entries 322 e) corresponding to database change indicators 230.

An advantage of the change log compaction data structure 324 is that itis an acceleration structure for finding modified data (i.e. changes) byexamining a subset of data space entries and/or index entries. Forexample, the change log compaction data structure 324 efficiently findsdata that has changed since a specific timestamp (e.g., version t). Thechange log compaction data structure 324 may be a space efficient datastructure because all missing data ranges (i.e. data ranges not includedin the structure) have been filtered (e.g., a bloom filter) to ensure aclient 20 directs a query 140 to predominantly modified data and/or datarepresentations. As an example, the synchronization system 100 isconfigured to use bloom filters to optimize fetching changed data toincorporate into the change log compaction data structure 324. Bloomfilters may be integrated with or operate in conjunction with thedatabase logger 200 and/or the compactor 300.

Additionally or alternatively, the change log compaction data structure324 may be limited to a finite number of version (e.g., the most recentversions) of the database 130 to further reduce query response time. Inother words, the change log compaction data structure 324 is a spaceconserving structure formed of data relevant to client synchronizationwith the database 130. Another advantage of the database synchronizationsystem 100 is that it may provide for adaptive granularity by providingprimitives to merge entries and permitting decision making about whatshould be merged and when. For example, merging entries may besimplified with negative space 328 known to have no changes to thedatabase 130.

FIGS. 4A-4D are examples of queries 140 to the database synchronizationsystem 100. In these examples, the compacter 300 further includes aquery analyzer 330. The query analyzer 330 is configured to receive thequery 140 for changes to the database 130 from the client 20. In someexamples, the query 140 indicates a set of data ranges 142 and aninterval of time 144. For example, referring back to FIG. 1B, the query140 implied an interval of time 144 by querying whether there had beenupdates since the last update at 9:47 A.M. In other words, the query 140itself implied an interval of time 144 between the 9:47 A.M. and thetime of the query 140. With the query 140, the query analyzer 330accesses the change log compaction data structure 324 constructed ofentries 322 by the constructer 320. Based on the entries 322 within thechange log compaction data structure 324, the query analyzer 330determines whether any entry 322 satisfies the query 140. When at leastone entry 322 of the change log compaction data structure 324 satisfiesthe query 140, the database synchronization system 100 returns thecorresponding at least one entry 322 to the client 20.

Referring to FIGS. 4A-4D, these figures are predominantly similarexamples of the compacter 300 except for the particular response to thequery 140. In some configurations, the database synchronization system100 returns the corresponding at least one entry 322 to the client 20 asthe response 150. Depending on the at least one entry 322 satisfying thequery 140, the response 150 may additionally include further detailregarding the at least one entry 322. In some examples, when multipleentries 322 of the change log compaction data structure 324 satisfy thequery 140, the database synchronization system 100 returns thecorresponding entries 322 to the client 20 as a sorted range 152 by astart of the range 152 in the index space 326 i or the data space 326 d.For example, by sorting the range 152, the client 20 may receive thesynchronization chronologically. In some implementations, the entry 322satisfying the query 140 is an interval entry 322 i. Here, for eachinterval entry 322 i satisfying the query 140, the response 150 includesa span 154 that is defined as a set of all corresponding versions of thedatabase 130 defined by the interval entry 322 i. In theseimplementations, as part of the span 154, the response 150 includes astart version t_(start) of the database 130 and an end version t_(end)of the database 130.

The response 150, as illustrated by FIG. 4A, may be at least one of adatabase change indicator 230, a database change log entry 210, or achange corresponding to the at least one entry 322. For example, thedatabase synchronization system 100 is configured to reverse determine,from the at least one entry 322 satisfying the query 140, thecorresponding database change indicator 230, the database change logentry 210, or the underlying change in the data D. In some examples,such as FIGS. 4B and 4D, the database synchronization system 100communicates a first response 150 a to the database logger 200. Thefirst response 150 a may be communicated by the compacter 300 and mayinclude either the at least one entry 322 satisfying the query 140, adatabase change indicator 230 corresponding to the at least one entry322, or both. If the database logger 200 receives, as the first response150 a, at least one database change indicator 230, the database logger200 may be further configured to cross reference the at least onedatabase change indicator 230 to provide, as a second response 150 b tothe client 20 and/or the client device 102, a database change log entry210 and/or the original data change.

FIGS. 4C and 4D are examples of the query 140 and the response 150 asthey relate to the examples illustrated in FIGS. 2 and 3. FIG. 4C is anexample of a single response 150 based on the query 140, while FIG. 4Dis an example of a multi-step response 150 based on the same query 140.FIGS. 4C and 4D depict the query 140 with a data range 142 of data D₁-D₅and a time interval 144 of t₀-t₃. Here, the query analyzer 330 receivesthe query 140 and compares the data range 142 and the time interval 144to the change log compaction data structure 324. This comparisonidentifies three overlapping entries 322 within the change logcompaction data structure 324—an interval entry 322 i and two rangeentries 322 r, 322 r ₁₋₂. As part of the response 150, the databasesynchronization system 100 returns a span 154, t₃-t₅, of the intervalentry 322 i along with a sorted range 152. FIG. 4D depicts similarresults, but with respect to a first response 150 a and a secondresponse 150 b. As illustrated, the first response 150 a returns entries322 i, 322 r ₁, 322 r ₂ and database change indicators 230 a-d. Here,the query analyzer 330 communicates the first response 150 a to thedatabase logger 200 such that the database logger 200 may additionallyprovide the corresponding database log entries 210 a-d and/or the dataD₁₋₂ and D₆₋₇ related to the database log entries 210 a-d. Thesecorresponding database log entries 210 a-d and/or data D₁₋₂ and D₆₋₇ maythen be communicated to the client 20 and/or the client device 102 viathe second response 150 b. In some examples, the database synchronizersystem 100 communicates both the first response 150 a and the secondresponse 150 b to the client 20 and/or the client device 102.

Referring back to FIG. 2, FIG. 2 further illustrates a potentialdeficiency with some systems that update according to database changes.For example, if a client device 102 previously updated at the firstversion t₀ and then subsequently updates at the fourth version t₃, insome systems the client device 102 may receive inaccurate communicationthat there have been no changes to the database 130. For example, if theclient device 102 was configured to scan for changes at the fourthversion t₃ compared to the first version t₀, the client device 102 mayfail to identify changes because the database 130 appears to contain thesame data D₁-D₅. The database 130, however, according to the databasechange log 220, has logged four modifications to the database 130.During this time, the database 130 has actually expanded to include dataD₆ and also later removed data D₆. In other words, comparing a firstsnapshot (e.g., first version at to) to a second snapshot (e.g., thefourth version at t₃) fails to account for the changes that occurredbetween the first snapshot and the second snapshot. If the client device102 instead reviewed the change log compaction data structure 324, theclient device 102 would identify changes between the first version t₀and the fourth version t₃. Knowledge of the changes that occur betweensnapshots may aid data troubleshooting, indicate unauthorizedmodification of data to a system, or identify for a client 20 whetherthe changes may impact other data.

FIGS. 5 and 6 are example methods 500 and 600 regarding operating adatabase synchronization system. At 502, the method 500 includesreceiving database change indicators 230 indicating changes to adatabase 130. At 504, the method 500 further includes, for each databasechange indicator 230, determining whether the respective change to thedatabase 130 corresponds to a single change 312 to a single version ofthe database 130, one of one of multiple changes 314 c to a commonversion of the database 130, or one of multiple changes 314 d todifferent versions of the database 130. At 506, the method 500 alsoincludes generating a change log compaction data structure 324comprising entries 322. At 602, the method 600 includes receiving aquery 140 for changes to a database 130 from a client 102, the query 140indicating a set of data ranges 142 and an interval of time 144. At 604,the method 600 includes accessing a change log compaction data structure324 comprising entries 322. At 606, the method 600 further includesdetermining whether any entry 322 of the change log compaction datastructure 324 satisfies the query 140. At 608, the method 600 alsoincludes when at least one entry 322 of the change log compaction datastructure 324 satisfies the query 140, returning the corresponding atleast one entry 322 to the client 20.

FIG. 7 is schematic view of an example computing device 700 that may beused to implement the systems and methods described in this document.The computing device 700 is intended to represent various forms ofdigital computers, such as laptops, desktops, workstations, personaldigital assistants, servers, blade servers, mainframes, and otherappropriate computers. The components shown here, their connections andrelationships, and their functions, are meant to be exemplary only, andare not meant to limit implementations of the inventions describedand/or claimed in this document.

The computing device 700 includes a processor 710, memory 720, a storagedevice 730, a high-speed interface/controller 740 connecting to thememory 720 and high-speed expansion ports 750, and a low speedinterface/controller 760 connecting to a low speed bus 770 and a storagedevice 730. Each of the components 710, 720, 730, 740, 750, and 760, areinterconnected using various busses, and may be mounted on a commonmotherboard or in other manners as appropriate. The processor 710 canprocess instructions for execution within the computing device 700,including instructions stored in the memory 720 or on the storage device730 to display graphical information for a graphical user interface(GUI) on an external input/output device, such as display 780 coupled tohigh speed interface 740. In other implementations, multiple processorsand/or multiple buses may be used, as appropriate, along with multiplememories and types of memory. Also, multiple computing devices 700 maybe connected, with each device providing portions of the necessaryoperations (e.g., as a server bank, a group of blade servers, or amulti-processor system).

The memory 720 stores information non-transitorily within the computingdevice 700. The memory 720 may be a computer-readable medium, a volatilememory unit(s), or non-volatile memory unit(s). The non-transitorymemory 720 may be physical devices used to store programs (e.g.,sequences of instructions) or data (e.g., program state information) ona temporary or permanent basis for use by the computing device 700.Examples of non-volatile memory include, but are not limited to, flashmemory and read-only memory (ROM)/programmable read-only memory(PROM)/erasable programmable read-only memory (EPROM)/electronicallyerasable programmable read-only memory (EEPROM) (e.g., typically usedfor firmware, such as boot programs). Examples of volatile memoryinclude, but are not limited to, random access memory (RAM), dynamicrandom access memory (DRAM), static random access memory (SRAM), phasechange memory (PCM) as well as disks or tapes.

The storage device 730 is capable of providing mass storage for thecomputing device 700. In some implementations, the storage device 730 isa computer-readable medium. In various different implementations, thestorage device 730 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device, a flash memory or other similarsolid state memory device, or an array of devices, including devices ina storage area network or other configurations. In additionalimplementations, a computer program product is tangibly embodied in aninformation carrier. The computer program product contains instructionsthat, when executed, perform one or more methods, such as thosedescribed above. The information carrier is a computer- ormachine-readable medium, such as the memory 720, the storage device 730,or memory on processor 710.

The high speed controller 740 manages bandwidth-intensive operations forthe computing device 700, while the low speed controller 760 manageslower bandwidth-intensive operations. Such allocation of duties isexemplary only. In some implementations, the high-speed controller 740is coupled to the memory 720, the display 780 (e.g., through a graphicsprocessor or accelerator), and to the high-speed expansion ports 750,which may accept various expansion cards (not shown). In someimplementations, the low-speed controller 760 is coupled to the storagedevice 730 and a low-speed expansion port 790. The low-speed expansionport 790, which may include various communication ports (e.g., USB,Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or moreinput/output devices, such as a keyboard, a pointing device, a scanner,or a networking device such as a switch or router, e.g., through anetwork adapter.

The computing device 700 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 700 a or multiple times in a group of such servers 700a, as a laptop computer 700 b, or as part of a rack server system 700 c.

Various implementations of the systems and techniques described hereincan be realized in digital electronic and/or optical circuitry,integrated circuitry, specially designed ASICs (application specificintegrated circuits), computer hardware, firmware, software, and/orcombinations thereof. These various implementations can includeimplementation in one or more computer programs that are executableand/or interpretable on a programmable system including at least oneprogrammable processor, which may be special or general purpose, coupledto receive data and instructions from, and to transmit data andinstructions to, a storage system, at least one input device, and atleast one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium” and“computer-readable medium” refer to any computer program product,non-transitory computer readable medium, apparatus and/or device (e.g.,magnetic discs, optical disks, memory, Programmable Logic Devices(PLDs)) used to provide machine instructions and/or data to aprogrammable processor, including a machine-readable medium thatreceives machine instructions as a machine-readable signal. The term“machine-readable signal” refers to any signal used to provide machineinstructions and/or data to a programmable processor.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby special purpose logic circuitry, e.g., an FPGA (field programmablegate array) or an ASIC (application specific integrated circuit).Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Computer readable media suitable for storingcomputer program instructions and data include all forms of non-volatilememory, media and memory devices, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks;magneto optical disks; and CD ROM and DVD-ROM disks. The processor andthe memory can be supplemented by, or incorporated in, special purposelogic circuitry.

To provide for interaction with a user, one or more aspects of thedisclosure can be implemented on a computer having a display device,e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, ortouch screen for displaying information to the user and optionally akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input. In addition, a computer can interact with a user bysending documents to and receiving documents from a device that is usedby the user; for example, by sending web pages to a web browser on auser's client device in response to requests received from the webbrowser.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. Accordingly, otherimplementations are within the scope of the following claims.

What is claimed is:
 1. A method comprising: receiving, at dataprocessing hardware, database change indicators indicating changes to adatabase among versions of the database, each version corresponding toan increment of a clock associated with a processor for the database;for each database change indicator, determining, by the data processinghardware, whether the respective change to the database corresponds to:a single change to a single version of the database; one of multiplechanges to a common version of the database; or one of multiple changesto different versions of the database; and generating, by the dataprocessing hardware, a change log compaction data structure comprisingentries, the change log compaction data structure comprising an indexspace and a data space, the index space comprising a version spacecorresponding to the versions of the database, each entry comprising: anexact entry defining a corresponding single change to a correspondingsingle version of the database; a range entry defining correspondingmultiple changes to a corresponding common version of the database; oran interval entry defining corresponding multiple changes tocorresponding different versions of the database, wherein any negativespace of the index space for the change log compaction data structurecomprises known space having no changes to the database.
 2. The methodof claim 1, wherein the database change indicator comprises a commit logentry.
 3. The method of claim 1, wherein the different versions of thedatabase corresponding to the interval entry span a continuous intervalof time.
 4. A method comprising: receiving, at data processing hardware,a query for changes to a database from a client, the query indicating aset of data ranges and an interval of time; accessing, by the dataprocessing hardware, a change log compaction data structure comprisingentries representing changes to a database among versions of thedatabase, each version corresponding to an increment of a clockassociated with a processor for the database, the change log compactiondata structure comprising an index space and a data space, the indexspace comprising a version space corresponding to the versions of thedatabase, wherein any negative space of the index space for the changelog compaction data structure comprises known space having no changes tothe database, and wherein each entry comprises: an exact entry defininga corresponding single change to a corresponding single version of thedatabase; a range entry defining corresponding multiple changes to acorresponding common version of the database; or an interval entrydefining corresponding multiple changes to corresponding differentversions of the database; determining, by the data processing hardware,whether any entry of the change log compaction data structure satisfiesthe query; and when at least one entry of the change log compaction datastructure satisfies the query, returning, by the data processinghardware, the corresponding at least one entry to the client.
 5. Themethod of claim 4, further comprising, when at least one entry of thechange log compaction data structure satisfies the query, returning, bythe data processing hardware, at least one of a database changeindicator, a database change log entry, or a change corresponding to theat least one entry of the client.
 6. The method of claim 4, furthercomprising, when at least one entry of the change log compaction datastructure satisfies the query, returning, by the data processinghardware, a bloom filter corresponding to the at least one entry to theclient.
 7. The method of claim 4, further comprising, when at least oneentry of the change log compaction data structure satisfies the query,returning, by the data processing hardware, any overlapping entries ofthe index space of the at least one entry satisfying the query, at orfollowing the at least one entry satisfying the query.
 8. The method ofclaim 4, further comprising, when multiple entries of the change logcompaction data structure satisfy the query, returning, by the dataprocessing hardware, the corresponding entries to the client as a rangesorted by a start of the range in the index space or the data space. 9.The method of claim 4, further comprising: determining, by the dataprocessing hardware, whether any entry of the change log compaction datastructure satisfying the query comprises an interval entry; and for eachinterval entry satisfying the query, returning a span defined as a setof all corresponding versions of the database defined by the intervalentry, the span having a start version of the database and an endversion of the database.
 10. A system comprising: data processinghardware; and memory hardware in communication with the data processinghardware, the memory hardware storing instructions that when executed onthe data processing hardware cause the data processing hardware toperform operations comprising: receiving database change indicatorsindicating changes to a database among versions of the database, eachversion corresponding to an increment of a clock associated with aprocessor for the database; for each database change indicator,determining, whether the respective change to the database correspondsto: a single change to a single version of the database; one of multiplechanges to a common version of the database; or one of multiple changesto different versions of the database; and generating a change logcompaction data structure comprising entries, the change log compactiondata structure comprising an index space and a data space, the indexspace comprising a version space corresponding to the versions of thedatabase, each entry comprising: an exact entry defining a correspondingsingle change to a corresponding single version of the database; a rangeentry defining corresponding multiple changes to a corresponding commonversion of the database; or an interval entry defining correspondingmultiple changes to corresponding different versions of the database,wherein any negative space of the index space for the change logcompaction data structure comprises known space having no changes to thedatabase.
 11. The system of claim 10, wherein the database changeindicator comprises a commit log entry.
 12. The system of claim 10,wherein the different versions of the database corresponding to theinterval entry span a continuous interval of time.
 13. A systemcomprising: data processing hardware; and memory hardware incommunication with the data processing hardware, the memory hardwarestoring instructions that when executed on the data processing hardwarecause the data processing hardware to perform operations comprising:receiving a query for changes to a database from a client, the queryindicating a set of data ranges and an interval of time; accessing achange log compaction data structure comprising entries representingchanges to a database among versions of the database, each versioncorresponding to an increment of a clock associated with a processor forthe database, the change log compaction data structure comprising anindex space and a data space, the index space comprising a version spacecorresponding to the versions of the database, wherein any negativespace of the index space for the change log compaction data structurecomprises known space having no changes to the database, and wherein,each entry comprises: an exact entry defining a corresponding singlechange to a corresponding single version of the database; a range entrydefining corresponding multiple changes to a corresponding commonversion of the database; or an interval entry defining correspondingmultiple changes to corresponding different versions of the database;determining whether any entry of the change log compaction datastructure satisfies the query; and when at least one entry of the changelog compaction data structure satisfies the query, returning thecorresponding at least one entry to the client.
 14. The system of claim13, further comprising, when at least one entry of the change logcompaction data structure satisfies the query, returning at least one ofa database change indicator, a database change log entry, or a changecorresponding to the at least one entry of the client.
 15. The system ofclaim 13, further comprising, when at least one entry of the change logcompaction data structure satisfies the query, returning a bloom filtercorresponding to the at least one entry to the client.
 16. The system ofclaim 13, further comprising, when at least one entry of the change logcompaction data structure satisfies the query, returning any overlappingentries of the index space of the at least one entry satisfying thequery, at or following the at least one entry satisfying the query. 17.The system of claim 13, further comprising, when multiple entries of thechange log compaction data structure satisfy the query, returning thecorresponding entries to the client as a range sorted by a start of therange in the index space or the data space.
 18. The system of claim 13,further comprising: determining whether any entry of the change logcompaction data structure satisfying the query comprises an intervalentry; and for each interval entry satisfying the query, returning aspan defined as a set of all corresponding versions of the databasedefined by the interval entry, the span having a start version of thedatabase and an end version of the database.